ChungMG

Stochastic Differential Equation (SDE)

A Stochastic Differential Equation (SDE) is a differential equation in which one or more terms is a stochastic process, resulting in a solution that is itself a stochastic process. SDEs are used to model systems with random fluctuations and are fundamental to [[Diffusion Model|Diffusion Models]], financial mathematics, and physics.

---

1. Core Concept

1.1 From ODE to SDE

Ordinary Differential Equation (ODE):

$$\frac{\mathrm{d}x}{\mathrm{d}t}=f(t,x)$$

Stochastic Differential Equation (SDE):

$$\mathrm{d}{x}_t=f(t,x_t)\mathrm{d}t+g(t,x_t)\mathrm{d}{W}_t$$

The key difference is the addition of the stochastic term $g(t,x_t)\mathrm{d}{W}_t$ , where $W_t$ is a [[Wiener Process|Wiener Process]].

[!NOTE] Key Insight
While ODEs describe deterministic evolution, SDEs incorporate random noise, making them suitable for modeling real-world systems with uncertainty.

1.2 Components of SDE

Component	Notation	Role
Drift coefficient	$f(t,x_t)$ or $\mu (t,x_t)$	Deterministic trend
Diffusion coefficient	$g(t,x_t)$ or $\sigma (t,x_t)$	Noise intensity
**[[Wiener Process	Wiener Process]]**	$W_t$ or $B_t$
State variable	$x_t$ or $X_t$	System state

---

2. Mathematical Foundation

2.1 Itô Interpretation

The SDE is interpreted as an integral equation:

$$x_t=x_0+{\int }_0^{t}f(s,x_s)\mathrm{d}s+{\int }_0^{t}g(s,x_s)\mathrm{d}{W}_s$$

where the second integral is an [[Itô Integral|Itô integral]].

2.2 Itô’s Lemma

Theorem: For an Itô process:

$$\mathrm{d}{x}_t={\mu }_t\mathrm{d}t+{\sigma }_t\mathrm{d}{W}_t$$

and a twice-differentiable function $h(t,x)$ , we have:

$$\mathrm{d}h(t,x_t)=(\frac{\partial h}{\partial t}+{\mu }_t\frac{\partial h}{\partial x}+\frac{1}{2}{\sigma }_t^2\frac{{\partial }^{2}h}{\partial x^2})\mathrm{d}t+{\sigma }_t\frac{\partial h}{\partial x}\mathrm{d}{W}_t$$

[!WARNING] Itô vs Stratonovich
The extra term $\frac{1}{2}{\sigma }_t^2\frac{{\partial }^{2}h}{\partial x^2}$ is unique to Itô calculus and arises from the non-zero quadratic variation of [[Wiener Process|Wiener process]]. This term does not appear in classical calculus.

2.3 Itô’s Lemma in Multiple Dimensions

For $x_t\in {\mathbb{R}}^n$ with SDE:

$$\mathrm{d}{x}_t={\mu }_t\mathrm{d}t+{\sigma }_t\mathrm{d}{W}_t$$

and function $h(t,x):\mathbb{R}\times {\mathbb{R}}^n\to {\mathbb{R}}^m$ :

$$\mathrm{d}h=(\frac{\partial h}{\partial t}+\sum _i{\mu }_t^i\frac{\partial h}{\partial x_i}+\frac{1}{2}\sum _{i,j}({\sigma }_t{\sigma }_t^{\mathrm{\top }}{)}_{ij}\frac{{\partial }^{2}h}{\partial x_i\partial x_j})\mathrm{d}t+\sum _i\frac{\partial h}{\partial x_i}{\sigma }_t^i\mathrm{d}{W}_t$$

---

3. Common Types of SDEs

3.1 Linear SDE

$$\mathrm{d}{x}_t=(a(t)x_t+b(t))\mathrm{d}t+(c(t)x_t+d(t))\mathrm{d}{W}_t$$

Solution method: Use integrating factor or variation of constants.

3.2 Geometric [[Wiener Process|Brownian Motion]]

$$\mathrm{d}{S}_t=\mu S_t\mathrm{d}t+\sigma S_t\mathrm{d}{W}_t$$

Solution (using Itô’s Lemma on $\log S_t$ ):

$$S_t=S_0\exp ((\mu -\frac{{\sigma }^2}{2})t+\sigma W_t)$$

Application: Stock price modeling (Black-Scholes).

3.3 Ornstein-Uhlenbeck Process

$$\mathrm{d}{x}_t=-\theta x_t\mathrm{d}t+\sigma \mathrm{d}{W}_t$$

Solution:

$$x_t=x_0{e}^{-\theta t}+\sigma {\int }_0^t{e}^{-\theta (t-s)}\mathrm{d}{W}_s$$

Properties:

• Mean-reverting to zero
• Stationary distribution: $x_{\mathrm{\infty }}\sim \mathcal{N}(0,\frac{{\sigma }^2}{2\theta })$
• Used in interest rate models (Vasicek model)

3.4 Variance-Preserving SDE (VP-SDE)

$$\mathrm{d}{x}_t=-\frac{1}{2}\beta (t)x_t\mathrm{d}t+\sqrt{\beta (t)}\mathrm{d}{W}_t$$

Properties:

• Marginal distribution: $x_t\mid x_0\sim \mathcal{N}(e^{-\frac{1}{2}{\int }_0^t\beta (s)ds}{x}_0,(1-e^{-{\int }_0^t\beta (s)ds})I)$
• Preserves variance in $[0,1]$
• Primary choice for [[Diffusion Model|Diffusion Models]]

3.5 Variance-Exploding SDE (VE-SDE)

$$\mathrm{d}{x}_t=\sqrt{\frac{\mathrm{d}{\sigma }^2(t)}{\mathrm{d}t}}\mathrm{d}{W}_t$$

Properties:

• Variance grows with time: $\text{Var}(x_t)={\sigma }^2(t)$
• No drift term
• Alternative formulation for diffusion models

---

4. Solving SDEs

4.1 Analytical Solutions

Some SDEs have closed-form solutions:

SDE	Solution	Method
$\mathrm{d}{x}_t=\mu \mathrm{d}t+\sigma \mathrm{d}{W}_t$	$x_t=x_0+\mu t+\sigma W_t$	Direct integration
$\mathrm{d}{S}_t=\mu S_t\mathrm{d}t+\sigma S_t\mathrm{d}{W}_t$	$S_t=S_0{e}^{(\mu -{\sigma }^2/2)t+\sigma W_t}$	Itô’s Lemma
$\mathrm{d}{x}_t=-\theta x_t\mathrm{d}t+\sigma \mathrm{d}{W}_t$	$x_t=x_0{e}^{-\theta t}+\sigma {\int }_0^t{e}^{-\theta (t-s)}\mathrm{d}{W}_s$	Integrating factor

4.2 Numerical Methods

Euler-Maruyama Method

Simplest discretization:

$$x_{t+\mathrm{\Delta }t}=x_t+f(t,x_t)\mathrm{\Delta }t+g(t,x_t)\sqrt{\mathrm{\Delta }t}ϵ,ϵ\sim \mathcal{N}(0,1)$$

Convergence: Strong order 0.5, weak order 1.0

Milstein Method

Higher-order method:

$$x_{t+\mathrm{\Delta }t}=x_t+f\mathrm{\Delta }t+g\mathrm{\Delta }W+\frac{1}{2}g\frac{\partial g}{\partial x}((\mathrm{\Delta }W{)}^2-\mathrm{\Delta }t)$$

Convergence: Strong order 1.0

# Pseudocode: Euler-Maruyama Method
def euler_maruyama(f, g, x0, T, N):
    dt = T / N
    x = x0
    path = [x0]
    
    for i in range(N):
        dW = sqrt(dt) * sample_normal(0, 1)
        x = x + f(t, x) * dt + g(t, x) * dW
        path.append(x)
    
    return path

---

5. Fokker-Planck Equation

5.1 Forward Kolmogorov Equation

The probability density $p(t,x)$ of the SDE solution satisfies:

$$\frac{\partial p}{\partial t}=-\frac{\partial }{\partial x}[f(t,x)p]+\frac{1}{2}\frac{{\partial }^2}{\partial x^2}[g(t,x{)}^{2}p]$$

This is the Fokker-Planck equation (or forward Kolmogorov equation).

5.2 Stationary Distribution

For time-homogeneous SDE $\mathrm{d}{x}_t=f(x_t)\mathrm{d}t+g(x_t)\mathrm{d}{W}_t$ , the stationary distribution $p_{\mathrm{\infty }}(x)$ satisfies:

$$0=-\frac{\mathrm{d}}{\mathrm{d}x}[f(x)p_{\mathrm{\infty }}(x)]+\frac{1}{2}\frac{{\mathrm{d}}^2}{\mathrm{d}{x}^2}[g(x{)}^2{p}_{\mathrm{\infty }}(x)]$$

Solution (for $g(x)=\sigma$ constant):

$$p_{\mathrm{\infty }}(x)\propto \exp (\frac{2}{{\sigma }^2}{\int }^{x}f(y)\mathrm{d}y)$$

5.3 Connection to [[Probability Flow ODE|Probability Flow ODE]]

The Fokker-Planck equation can be written as a continuity equation:

$$\frac{\partial p}{\partial t}=-\mathrm{\nabla }\cdot (vp)$$

where the velocity field $v$ defines the [[Probability Flow ODE]] with the same marginals.

---

6. SDEs in Diffusion Models

6.1 Forward Process

In diffusion models, data $x_0$ is gradually corrupted by noise:

$$\mathrm{d}{x}_t=f(t)x_t\mathrm{d}t+g(t)\mathrm{d}{W}_t$$

Common choices:

SDE Type	$f(t)$	$g(t)$	Marginal
VP-SDE	$-\frac{1}{2}\beta (t)$	$\sqrt{\beta (t)}$	$\mathcal{N}(e^{-\frac{1}{2}\int \beta }{x}_0,(1-e^{-\int \beta })I)$
VE-SDE	$0$	$\sqrt{\beta (t)}$	$\mathcal{N}(x_0,{\int }_0^t\beta (s)ds\cdot I)$
Sub-Variance	$-\beta (t)$	$\sqrt{2\beta (t)}$	Interpolates between VP and VE

6.2 Reverse Process

The time-reversal of an SDE (Anderson, 1982):

$$\mathrm{d}{x}_t=[f(t)x_t-g(t{)}^2{\mathrm{\nabla }}_x\log p_t(x)]\mathrm{d}t+g(t)\mathrm{d}{\overline{W}}_t$$

where:

• ${\overline{W}}_t$ is reverse-time [[Wiener Process|Wiener process]]
• ${\mathrm{\nabla }}_x\log p_t(x)$ is the [[Score Function|score function]]
• Must be solved backward from $t=T$ to $t=0$

6.3 Score Matching

Learn the score function with neural network $s_{\theta }(x,t)\approx {\mathrm{\nabla }}_x\log p_t(x)$ :

Objective:

$$\mathcal{L}(\theta )={\mathbb{E}}_{t,x_0,x_t}[\parallel s_{\theta }(x_t,t)-{\mathrm{\nabla }}_{x_t}\log p(x_t\mid x_0){\parallel }^2]$$

For VP-SDE:

$${\mathrm{\nabla }}_{x_t}\log p(x_t\mid x_0)=-\frac{x_t-e^{-\frac{1}{2}{\int }_0^t\beta (s)ds}{x}_0}{1-e^{-{\int }_0^t\beta (s)ds}}$$

---

7. Beyond SDE: ODE Equivalence and Fast Sampling

7.1 From SDE to [[Probability Flow ODE]]

The reverse SDE can be deterministically transformed into an ODE with identical marginals:

$$\mathrm{d}x=[f(t)x-\frac{1}{2}g(t{)}^2{\mathrm{\nabla }}_x\log p_t(x)]\mathrm{d}t$$

This [[Probability Flow ODE]] enables:

• Deterministic sampling (same noise → same output)
• Exact likelihood computation via instantaneous change of variables
• Inversion of real data to latent space

7.2 Fast SDE Sampling Methods

Method	Type	Steps	Key Idea
DDPM	SDE	1000	Original Markov chain
[[DDIM]]	Non-Markovian	50-100	Relaxes Markov assumption
[[DPM-Solver]]	ODE	10-20	Semi-linear structure exploitation
Predictor-Corrector	SDE+ODE	20-50	Langevin refinement steps

7.3 Numerical Stability in SDE Solvers

Key challenges for diffusion model SDEs:

1. Stiffness near $t=0$ : VP-SDE becomes very stiff

   • Fix: Non-uniform time discretization, more steps near $t=0$

2. Score function explosion: As $t\to 0$ , the score ${\mathrm{\nabla }}_x\log p_t(x)$ diverges

   • Fix: Use $ϵ$ -prediction instead of direct score prediction

3. Discretization error accumulation: Euler-Maruyama error propagates

   • Fix: Higher-order methods (Milstein, [[DPM-Solver]])

4. SDE vs ODE trade-off:

   • SDE: Better quality (injecting fresh noise), slower
   • ODE: Faster, deterministic, better for inversion

8. Existence and Uniqueness

8.1 Lipschitz Conditions

Theorem: If $f(t,x)$ and $g(t,x)$ satisfy:

1. Lipschitz condition: $|f(t,x)-f(t,y)|+|g(t,x)-g(t,y)|\le K|x-y|$
2. Linear growth: $|f(t,x){|}^2+|g(t,x){|}^2\le K(1+|x{|}^2)$

then the SDE has a unique strong solution.

8.2 Weak vs Strong Solutions

Type	Definition	Requirement
Strong solution	$x_t$ adapted to filtration of $W_t$	Fixed probability space
Weak solution	Existence of $(x_t,W_t)$ jointly	Distribution equivalence

---

9. Girsanov Theorem

9.1 Change of Measure

Theorem: Under suitable conditions, we can change measure $\mathbb{P}\to \mathbb{Q}$ to eliminate drift:

If $\mathrm{d}{x}_t={\mu }_t\mathrm{d}t+{\sigma }_t\mathrm{d}{W}_t^{\mathbb{P}}$ , then under $\mathbb{Q}$ :

$$\mathrm{d}{x}_t={\sigma }_t\mathrm{d}{W}_t^{\mathbb{Q}}$$

where $\mathrm{d}{W}_t^{\mathbb{Q}}=\mathrm{d}{W}_t^{\mathbb{P}}+\frac{{\mu }_t}{{\sigma }_t}\mathrm{d}t$ .

9.2 Applications

• Risk-neutral pricing in finance
• Importance sampling for variance reduction
• Likelihood ratio computation in diffusion models

---

10. Core Formula Cards

[!QUOTE] General SDE

$$\mathrm{d}{x}_t=f(t,x_t)\mathrm{d}t+g(t,x_t)\mathrm{d}{W}_t$$

[!QUOTE] Itô’s Lemma

$$\mathrm{d}h=(\frac{\partial h}{\partial t}+f\frac{\partial h}{\partial x}+\frac{1}{2}{g}^2\frac{{\partial }^{2}h}{\partial x^2})\mathrm{d}t+g\frac{\partial h}{\partial x}\mathrm{d}{W}_t$$

[!QUOTE] Geometric [[Wiener Process|Brownian Motion]]

$$\mathrm{d}{S}_t=\mu S_t\mathrm{d}t+\sigma S_t\mathrm{d}{W}_t$$

[!QUOTE] Ornstein-Uhlenbeck Process

$$\mathrm{d}{x}_t=-\theta x_t\mathrm{d}t+\sigma \mathrm{d}{W}_t$$

[!QUOTE] VP-SDE (Diffusion Models)

$$\mathrm{d}{x}_t=-\frac{1}{2}\beta (t)x_t\mathrm{d}t+\sqrt{\beta (t)}\mathrm{d}{W}_t$$

[!QUOTE] Reverse-Time SDE

$$\mathrm{d}{x}_t=[f(t)x_t-g(t{)}^2{\mathrm{\nabla }}_x\log p_t(x)]\mathrm{d}t+g(t)\mathrm{d}{\overline{W}}_t$$

[!QUOTE] Fokker-Planck Equation

$$\frac{\partial p}{\partial t}=-\frac{\partial }{\partial x}[fp]+\frac{1}{2}\frac{{\partial }^2}{\partial x^2}[g^{2}p]$$

[!QUOTE] Euler-Maruyama Discretization

$$x_{t+\mathrm{\Delta }t}=x_t+f(t,x_t)\mathrm{\Delta }t+g(t,x_t)\sqrt{\mathrm{\Delta }t}ϵ$$

---

Related Concepts

• [[Wiener Process|Wiener Process]]
• [[Itô Integral]]
• [[Itô’s Lemma]]
• [[Martingale]]
• [[Diffusion Model]]
• [[Probability Flow ODE]]
• [[Score Function]]
• [[DDIM]]
• [[DPM-Solver]]
• [[Flow Matching]]
• [[Markov Process]]
• [[Fokker-Planck Equation]]
• [[Kolmogorov Equations]]
• [[Langevin Dynamics]]
• [[Stochastic Process]]
• [[Brownian Motion]]

---

Dataview Query

LIST
FROM #sde OR #stochastic_calculus
SORT file.ctime DESC

---

References

• Book: Stochastic Differential Equations - Bernt Øksendal
• Book: [[Wiener Process|Brownian Motion]] and Stochastic Calculus - Karatzas & Shreve
• Paper: Score-Based Generative Modeling through SDEs (Song et al., 2021)
• Paper: Maximum Likelihood Training of Score-Based Diffusion Models (Song et al., 2021)
• Course: MIT 18.S096 Topics in Mathematics with Applications in Finance
• Course: CS236 Deep Generative Models (Stanford)